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1 . INTRODUCTION 

This  report  considers  a problem  of  selecting  from  a set 
of  objects  a subset  of  maximum  value  where  there  are  sequence 
dependent  constraints  which  restrict  the  choices  available. 

There  is  a natural  ordering  among  the  objects  which  dictates 
the  order  in  which  they  must  be  considered.  It  is  assumed 
that  the  decision  maker  has  complete  knowledge  of  the  opportu- 
nities which  will  arise  so  that  the  problem  is  one  of  determining 
which  opportunities  to  select  and  which  to  forego  in  preparation 
for  later  gain.  The  problem  in  which  the  opportunities  have 
some  probability  of  vanishing  without  notice  is  considered 
briefly,  but  the  solution  method  is  the  same  as  long  as  these 
probabilities  are  known. 

The  problem  can  be  cast  in  several  forms.  Reference  1 
describes  the  problem  in  the  context  of  an  "investigator"  pass- 
ing through  a region  in  which  there  is  a number  of  points  to 
be  examined.  It  cam  also  be  viewed  as  a "delivery"  problem 
in  which  a delivery  truck  has  a list  of  locations  and  delivery 
times.  If  a delivery  is  made,  it  must  be  done  at  the  given 
time  and  the  specified  location.  Knowing  the  transit  times 
from  point  to  point  and  the  value  of  each  customer,  the  problem 
is  to  select  the  most  valuable  subset  of  customers  to  serve. 

The  problem  can  be  viewed  as  a job  shop  scheduling 
problem  in  which  the  production  manager  has  a set  of  possible 
jobs  to  perform.  Each  job  has  a required  starting  time  and 
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known  set-up  and  processing  time.  The  problem  is  to  select 
the  most  valuable  subset  for  processing.  Reference  2 deals 
with  the  application  of  dynamic  programming  to  a similar  problem 
in  which  each  job  has  an  availability  interval  within  which  it 
must  be  processed  if  selected. 

The  problem  is  viewed  here  as  one  in  which  a defender  is 
guarding  some  region  against  attack  from  a set  of  attackers 
whose  times  and  points  of  arrival  at  the  region  are  known.  The 
defender  is  constrained  to  remain  on  the  boundary  of  the  region 
and  must  move  to  the  point  of  intrusion  at  the  time  the  attacker 
arrives  in  order  to  destroy  the  attacker.  This  is  equivalent 
to  saying  that  the  defender  can  not  engage  the  target  until  it 
crosses  some  threshold  but  it  must  be  engaged  immediately  there- 
after. The  selection  problem  arises  because  the  defender  is 
limited  in  the  rate  at  which  he  can  move  along  the  boundary. 

For  convenience  the  boundary  is  assumed  to  be  a straight 

line  segment  along  which  the  defender  moves.  Other  conf igurations 
are  possible.  For  example,  some  problems  in  point  defense  can 
be  formulated  in  the  same  way.  In  that  case  the  boundary  can 
be  thought  of  as  a circle  (of  zero  radius)  and  the  position  of 
the  defensive  system  is  represented  oy  its  angular  rotation 
or  orientation.  Figure  1 illustrates  a problem  of  this  type 
when  the  region  defined  is  the  portion  of  the  x-axis  from 


o B position 

FIGURE  1.  Illustration  of  Basic  Problem 
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It  is  convenient  to  think  of  all  the  attackers  as 
having  the  same  speed  and  travelling  on  parallel  courses 
directly  toward  the  boundary,  but  it  is  not  necessary  to  do  so. 

In  fact,  all  that  is  required  is  the  time  and  point  of  arrival 
on  the  boundary  for  each  attacker.  It  is  assumed  hereafter 
that  the  attackers  are  numbered  in  order  of  arrival  at  the 
boundary  and  that  the  times  and  points  of  arrival  are  known. 

Let  tj  be  the  time  of  arrival  of  attacker  j and  p ^ 
be  the  point  of  arrival  where  0 1.  Pj  £.  B.  The  value  of 
attacker  j is  represented  by  v^.  It  is  also  convenient 
to  define  a dummy  point  pQ , tg  with  value  vQ  ■ 0 representing 
the  initial  location  of  the  defender. 

For  visual  presentation  of  the  problem  a slight  trans- 
formation is  convenient.  This  problem  is  equivalent  to  one  in 
which  the  attackers  are  stationary  and  the  boundary  moves  in 
time  through  the  attackers.  As  an  example,  the  defender's 
position  in  the  two-dimensional  space  (x,t)  is  shown  in 
Figure  2. 

The  problem  remains  one  of  determining  the  optimal  set 
of  targets  for  the  defenders  to  engage  subject  to  limitations 
on  his  ability  to  reposition  his  defensive  system. 

This  report  deals  primarily  with  the  problem  in  which 
there  are  two  identical  defenders  although  the  ideas  are 
applicable  for  the  M-defender  problem.  A straightforward 
dynamic  programming  approach  for  the  M-defender  problem  is 
discussed  first.  See  also  Reference  1.  Next  an  efficient 
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solution  method  for  the  problem  with  a single  defender  is 
considered.  Then  an  improved  formulation  for  the  two-defender 
problem  is  presented  and  it  is  shown  how  that  problem  can  be 
solved  with  a greatly  reduced  computational  burden.  A computer 
problem  was  written  to  test  the  method  and  the  solution  to  a 
sample  problem  with  N ■ 50  attackers  is  included. 

Several  generalizations  are  discussed  in  Section  5. 


2.  DYNAMIC  PROGRAMMING  SOLUTION  FOR  M-DEFENDERS 

The  M investigator  problem  can  be  solved  with  a 
standard  dynamic  programming  approach  using  M state  variables. 
Define  the  stages  such  that  stage  n corresponds  to  the 
time  tn,  and  let  '^Mn^  ^e  state  variable 

describing  the  location  along  the  boundary  of  each  investigator 
at  stage  n.  The  return  function  for  stage  n is 


- 

W - ) 

( 0 


xin 


Pn  * any  i 
otherwise . 


The  stage  transformation  functions  can  be  written  as 


x.,,  3 t(  x,  D) 
n+l  n n n 


where  D^  is  the  vector  of  decisions  prescribing  the  heading 
of  all  investigators  as  they  move  from  stage  n to  n+l  and 
tn  is  the  function  which  describes  the  resulting  locations 
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at  stage  n+1.  The  decisions  5^  are  restricted  to  lie  in 

some  set  Kn-  An  alternative  formulation  simply  defines  5n 

as  the  set  of  points  to  which  the  investigators  move  so  that 

x_  , • t (x  , 5 ) • 5 , but  still  5 is  constrained  to  lie 
n+l  n n n n n 

in  some  set  of  feasible  solutions,  say  K . 

n 

The  complete  problem  can  be  written  as 


N 

maximize 

l r (.x  ,6  ) 

n«0  n n 


•ubj.ct  to 

n • 0 , . . . , N 

5n  c "n  • 

n - 0 , . . . , N 

and  can  be  solved  using  a standard  dynamic  programming  approach 
with  tabular  computations  once  a suitable  grid  has  been 
established  for  the  state  and  decision  variables.  The  solution 
requires  the  evaluation  of  the  following  recursive  equations 
beginning  with  stage  N 

Ws-i1  * 0 


VV  ■ tcn(VV  * n-° N’ 

Dncltn 


where 


*n.l  ’ VVBn’- 
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This  method  is  very  general  and  can  be  used  to  solve 
several  variations  of  the  basic  problem,  except  that  it 
becomes  computationally  unwieldy  as  M increases.  The  amount 
of  computation  required  rises  exponentially  with  M but 
linearly  in  N. 

Two  generalizations  that  can  be  solved  using  a similar 
formulation  are  the  problems  in  which  there  is  a restriction 
for  each  defender  on  the  total  number  of  attackers  it  can 
engage  or  a restriction  on  the  total  lateral  movement  of 
the  defender.  In  these  problems  the  vector  of  state  variables 
must  be  enlarged  to  include  a component  for  the  number  of 
engagements  remaining  and  a component  for  the  amount  of  lateral 
movement  remaining. 

The  formulation  above  is  not  practical  for  large 
values  of  M nor  for  problems  with  constraints  which  lead 
to  additional  state  variables,  and  the  purpose  of  this  report 
is  to  present  a method  which  is  more  efficient  them  that  just 
given,  but  first  an  efficient  method  for  the  basic  one-state- 
variable  problem  is  presented. 
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3.  EFFICIENT  SOLUTION  FOR  ONE  JE  FENDER 

For  the  problem  with  a single  defender,  as  illustrated 

in  Figure  1,  the  algorithm  shown  in  Figure  3 provides  an 

efficient  solution  procedure.  That  algorithm  assumes  a dummy 

point  at  Pq  having  value  v^  « 0 representing  the  starting 

location  of  the  defender.  The  computation  produces  in  reverse 

order  the  sequence  of  optimal  return  functions  f n , n = 0,...,N 

where  the  quantity  fn  represents  the  optimal  total  return 

that  caui  be  obtained  from  the  remaining  stages,  not  including 

n,  given  that  the  Dosition  of  the  defender  is  d at  time  t . 

* n n 

The  sequence  of  optimal  decisions  is  also  produced  auid  can  be 
used  beginning  at  d^  to  trace  the  optimal  policy.  The 
algorithm  relies  on  the  fact  that  the  only  relevant  value  of 
the  state  variable  at  any  stage  is  the  value  corresponding  to 
the  location  of  the  attacker  and  that  the  return  functions 
need  not  be  computed  for  any  other  values.  For  that  reason 
the  state  variable  is  suppressed  in  f^. 

This  algorithm  earn  easily  be  implemented  in  FORTRAN  as 
two  nested  DO  loops. 


4.  IMPROVED  FORMULATION  FOR  THE  TWO-DEFENDER  PROBLEM 

This  section  shows  how  an  improved  formulation  for  the 
problem  with  two  defenders  earn  be  obtained.  In  this  section  the 
N-attacker  problem  is  still  viewed  as  an  N stage  dynaunic 
programming  problem  where  stage  n corresponds  to  the  time  t 
at  which  the  defender  departs  from  the  target  n.  In  this 


9 


FIGURE  3.  Algorithm  for  the  problem  with  a single  defender 


formulation,  as  in  the  formulation  of  Section  3,  passing  through 
stage  n corresponds  to  engaging  target  nj  and  if  target  n is 
not  engaged,  then  stage  n is  bypassed. 

At  stage  n the  state  variables  xni , x^  represent 
the  positions  of  defenders  1 and  2 along  the  boundary.  For 
computations  it  is  assumed  that  a grid  is  established  for  the 
state  variables.  For  convenience  this  grid  is  assumed  to  be  the 
same  at  each  stage,  consisting  of  S+l  points  equally  spaced  a 
distance  ^ apart.  It  is  also  convenient  to  assume  that  the 
targets  occur  at  the  grid  points,  although  this  is  not  essential. 
If  they  were  located  between  grid  points  it  could  be  assumed  that 
a target  is  engaged  if  the  defender  passes  within  a distance  r 
of  the  target,  or  the  grid  points  could  be  redefined  at  each 
stage  to  include  the  target  location. 

One  observation  which  considerably  reduces  the  amount 

of  computation  is  that  if  the  defenders  pass  through  stage  n,  then 

one  of  the  defenders  must  be  located  at  p , the  location  of  the 

n 

n attacker.  Thus  instead  of  (S+l)*  possible  values  of 
the  state  variables  xnl  and  xfi2  we  need  to  consider  only 
(S+l)  + (S+l)  - 1 ■ 2S  + 1 values.  The  computational  reduction 
from  this  observation  is  roughly  equivalent  to  reducing  the 
number  of  state  variables  by  one. 

A second  observation  can  also  be  made.  This  is  that 
there  is  an  optimal  solution  in  which  the  two  defenders  do  not 
cross.  This  is  similar  to  the  observation  in  the  travelling 
salesman  problem  that  for  problems  with  Euclidean  distance 
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the  optimal  tour  need  not  cross  itself.  The  validity  of  this 
observation  can  be  established  by  noting  that  if  a point  of 
crossing  occurs,  the  defenders  could  simply  be  renumbered 
so  that  number  1 remains  on  the  left  and  number  2 on  the  right. 
This  follows  from  the  fact  that  the  defenders  are  identical, 
and  it  would  not  hold  if  they  were  able  to  move  at  different 
rates  along  the  boundary.  It  would  also  be  invalid  if  a penalty 
were  assessed  for  changing  the  direction  of  motion  from  left 
to  right  or  vice-versa  or  if  they  had  constraints  on  the  number  of 
attackers  handled. 


With  this  second  observation  the  number  of  possible 


values  of  the  state  variables  is  reduced  even  further.  The 
possible  values  of  the  state  variables  at  stage  n,  and 

xn2 * are  given  in  Table  1 where  is  the  x-coordinate  of 

the  target.  Since  the  targets  were  initially  assumed  to 

lie  on  the  grid  points  pn  - KA  for  some  K.  The  state  variable 
combinations  listed  in  Table  1 have  been  numbered  for  reference, 
bi t notice  that  knowing  the  state  number  is  equivalent  to 
having  the  value  of  both  state  variables.  For  example,  state 
number  K'  implies  that 


and 


nl 


n2 


if 


if 


K’d  < p. 
K'A  > p. 

K'A  < Pi 
K'A  > pL  . 
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State 

Number 

xnl 

xn2 

0 

0 

Pn 

1 

A 

Pn 

• 

• 

• 

K-l 

(K-l) A 

Pn 

K 

Pn 

(K  + l ) A 

S 

Pn 

• 

• 

(S+l) A 

TABLE  1 

Possible  values  of  the  state  variable 
in  the  two  defender  problem. 

» 


Thus  all  the  state  variable  information  is  conveyed  by  the 

state  numbers  and  it  is  unnecessary  to  write  both  state 

variables.  For  this  reason  it  is  adequate  to  let  the  single 

state  variable  at  stage  n be  the  state  number  SR  so  that 

S * 0,1,2,...,  S.  The  result  of  this  second  observation  is  to 
n 

reduce  the  number  of  state  variable  combinations  at  each  stage 
from  2S+1  to  S+l. 

In  the  dynamic  programming  formulation  fn(Sn)  denotes 
the  maximum  total  return  from  the  remaining  stages  n+l,...,N 
given  that  the  defenders  are  at  stage  n in  state  Sn*  The 
general  recursion  can  be  written  as 


max  {v.  + f.  (D  )}  , 

<S  , D ^n  Sn  n 
n n 


5n  > n'  Dn  € Ks  (Sn),  n = 1,...,N-1 
n 

where  5^  is  a decision  variable  denoting  the  stage  to  which 

the  defender  moves  next  and  is  the  decision  variable  which 

n 

determines  the  state  number  to  which  each  defender  moves  at 

stage  $n.  The  set  of  feasible  states  at  stage  5n  given 

state  S„  at  stage  n is  denoted  by  K.  (S  ) . 
n 5n  n 

From  state  Sn  it  may  not  be  possible  to  go  to  some 
stages  and,  among  those  that  are  possible, the  question  of  which  to 
enter  must  be  resolved.  The  general  computation  proceeds  as 
shown  in  Figure  4 beginning  at  stage  N-l.  Figure  4 assumes 
completion  of  the  computation  of  fn(Sn)  for  n 3 N-2,...,i+l 

so  that  the  next  step  is  to  evaluate  f^(0. 


14 


5.  MODIFICATIONS  TO  THE  BASIC  PROBLEM 

A similar  problem  in  which  each  target  j has  some 
probability  1 - p^  of  vanishing  before  it  is  engaged  can 
be  solved  by  the  same  method.  The  objective  becomes  maximization 
of  expected  value  and  the  computation  is  modified  by  diminishing 
the  value  of  target  j from  v^  to  Pjvj*  After  this  change, 
the  remainder  of  the  solution  method  is  the  same. 

Several  other  modifications  of  the  basic  problem  are  of 
interest.  These  include  the  problem  in  which  there  is  a 
limitation  on  the  total  motion  of  the  defender  as  would  be  the 
case  where  fuel  or  some  other  resource  is  consumed.  Another 
interesting  problem  arises  when  there  is  a constraint  on  the 
total  number  of  attackers  which  can  be  engaged  by  each  defender. 
Both  of  these  modifications  can  be  handled  in  the  basic  dynamic 
programming  formulation  by  the  addition  of  another  state  vari- 
able; but  as  mentioned  previously  large  problems  require  that 
more  efficient  methods  be  found. 

In  the  computer  program  which  implements  this  solution 
method  for  the  two-defender  problem  a further  savings  is 
incorporated.  If  the  difference  between  <5^  and  i is 
large,  or  at  least  if  the  time  difference  between  the  stages  is 
large,  it  is  almost  certainly  possible  to  engage  some  target 
between  i and  6^,  say  at  stage  j,  and  still  move  to  any 
desired  state  at  stage  <S^.  In  such  a case  it  is  unnecessary 
for  the  recursive  procedure  to  examine  stages  6^,  N 

as  possible  choices  of  the  next  stage  following  i.  This  is 
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because  those  choices  are  dominated  by  another  choice,  namely, 
moving  to  stage  j from  i.  A test  to  this  effect  was  incorporated 


with  a considerable  reduction  in  computer  time.  The  usefulness 
of  such  a test  depends  on  the  particular  rules  of  motion  assumed 
for  the  defender  and  will  be  pursued  here. 

Reference  3 considers  the  problem  from  the  point  of  view 
of  two  attackers  moving  through  a region  containing  N targets. 
Each  attacker  must  select  a subset  of  the  targets  and  no  target 
can  be  selected  by  both  attackers.  The  case  in  which  the 
attackers  enter  the  region  from  different  directions  is  considered. 
That  case  is  not  covered  here  since  the  ordering  of  the  stages 
differs  for  the  two  attackers. 


6 . COMPUTER  PROGRAM 

A program  was  written  in  FORTRAN  IV  to  implement  the 
procedure  described  in  Section  4 for  the  two  defender  problem. 
This  program  was  used  to  solve  several  problems  in  which  the 
attacker's  locations  were  generated  randomly  using  a uniform 
distribution  to  determine  the  coordinates  x and  y,  where 
0 <_  x < 11  and  0 £ t £ 100.  In  order  to  make  the  x-coordinates 
fall  on  a conveniently  spaced  grid,  the  x-values  generated  were 
truncated  to  an  integer  before  solving  the  problem. 

Table  2 shows  the  x and  t coordinates  for  a sample 
problem  involving  50  attackers.  Also  shown  are  the  randomly 
generated  target  values  ranging  from  1 to  5 . 
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A graphical  presentation  of  the  sample  problem  is 
given  in  Figure  5.  The  numbers  beside  each  point  are  the 
target  values  on  the  right  and  the  point  number  (mod  10)  on 
the  left.  The  two  lines  show  the  sets  of  attackers  engaged 
by  the  two  defenders.  The  roles  of  motion  used  in  this  sample 
problem  permitted  the  defenders  to  move  left  or  right  at  the 
rate  of  one  unit  per  unit  time.  The  scale  used  in  plotting 
Figure  5 is  compressed  in  the  t direction  and  it  gives  the 
appearance  that  the  defenders  can  move  more  rapidly  than  one 
unit  per  unit  time,  but  reference  to  the  coordinates  of  each 
point  will  confirm  that  the  rules  of  motion  are  not  violated. 
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1 

5.00 

0.10 

1 

2 

6.00 

2.33 

4 

3 

6.00 

5.20 

1 

4 

4.00 

5.53 

2 

5 

7.00 

7.74 

2 

6 

6.00 

9.96 

3 

7 

6.00 

9 .31 

1 

9 

4.00 

16.19 

3 

9 

3.00 

17.19 

4 

10 

4.00 

20.99 

4 

11 

2.00 

20.90 

5 

12 

6.00 

21.37 

1 

13 

4.00 

22.19 

4 

14 

9.00 

22.91 

5 

15 

1.00 

26.16 

2 

16 

9.00 

26.91 

3 

17 

6 .00 

31.96 

1 

19 

1.00 

32.69 

5 

19 

7.00 

42.19 

4 

20 

7.00 

43.04 

3 

21 

3.00 

44.45 

2 

22 

4.00 

45  .69 

2 

23 

5.00 

49.25 

5 

24 

7.00 

50.33 

3 

25 

9 .00 

53.22 

5 

26 

2.00 

55.22 

2 

27 

7.00 

59.42 

1 

29 

3.00 

59.99 

5 

29 

2.00 

60.39 

3 

30 

9.00 

60.63 

4 

31 

9.00 

65.35 

2 

32 

6.00 

65.54 

5 

33 

9.00 

66.17 

3 

34 

3.00 

65.79 

2 

35 

3.00 

66.99 

2 

36 

2.00 

69  .05 

2 

37 

9.00 

70.96 

3 

39 

4.00 

72.45 

5 

39 

0.00 

73.54 

4 

40 

9.00 

75.19 

3 

41 

6.00 

76.55 

3 

42 

4.00 

76.92 

2 

43 

7.00 

90 .05 

2 

44 

0 .00 

91.95 

2 

45 

4.00 

94.41 

1 

46 

9.00 

95  .02 

3 

47 

6.00 

90.00 

1 

49 

7.00 

90.34 

5 

49 

9.00 

92.90 

1 

50 

4.00 

99.97 

5 

Table  2.  Data  for  Sample  Problem  Involving  50  Attackers 
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